Meta-analysis selection bias in marketing research

a r t i c l e i n f o The tendency of meta-analytic authors to select particular studies is called selection bias. Selection bias can affect the strength of the meta-analytic estimate and the attention that scholars devote to the results. This research is, in effect, a meta-analysis of the effect sizes reported or calculated from 94 meta-analysis studies of various topics in marketing research. The analysis reveals that estimates depend on the publication status of the included studies. The greater the percentage of studies that were published in academic journals vs. non-published studies, the greater is the size of the meta-effects, and the more published studies from leading journals the meta-analysis includes, the stronger the effect size. The meta-analytic effect size is a mediator for the influence of both the ratio of unpublished studies and the ratio of studies from leading journals on the probability of a meta-analysis to be published in a leading journal, which increases the number of citations to a meta-analysis. The findings of this study have several implications for meta-analysts, editors, reviewers and the marketing community on how to conduct and read current and future meta-analysis in marketing research. It is a well-known problem that preferential publication of significant and strong results over non-significant and weak results leads to a literature that provides a false impression regarding the size of the effect in question. There is strong evidence from several fields of science that this " publication bias " exists (Dickersin, 2005). By including published and unpublished studies in their quantitative review, meta-analysts try to mitigate the problem that the publication status of a study (i.e., whether the study is published or unpublished) is related to the effect size estimate in the study. The efforts to include studies of various publication statuses and the tendency of meta-analytic authors to select particular studies—whether intentionally or not—are called selection biases (Ferguson & Brannick, 2012). The current study identifies and examines selection bias in 94 meta-analyses in marketing research and its consequences for academia. Selection bias arises from the selection decision of a meta-analyst, whereas publication bias is based on the decision of authors and editors to submit and to publish a manuscript, which precedes the selection decision of the meta-analyst. Although it is a different kind of bias, a selection bias might have similar consequences as a publication bias because certain studies are more likely …